Columbia Kermit/kermit.zip/newsgroups/misc.19981211-19990422/[e-mail] _Wed Mar 10 20:06:45 1999.msg

Internet Message Format | 2020-01-01 | 7KB

Return-Path: <[e-mail]>
Received: from newsmaster.cc.columbia.edu (newsmaster.cc.columbia.edu [128.59.59.30])
    by watsun.cc.columbia.edu (8.8.5/8.8.5) with ESMTP id UAA23494
    for <[e-mail]>; Wed, 10 Mar 1999 20:06:43 -0500 (EST)
Received: (from news@localhost)
    by newsmaster.cc.columbia.edu (8.8.5/8.8.5) id TAA10102
    for [e-mail]; Wed, 10 Mar 1999 19:50:29 -0500 (EST)
X-Authentication-Warning: newsmaster.cc.columbia.edu: news set sender to <news> using -f
From: [e-mail]
Subject: C-Kermit Scripting using INPUT and LOG SESSION
Date: 10 Mar 1999 19:42:53 -0500
Organization: Amherst College
Message-ID: <[e-mail]>
To: [e-mail]

Hi there.

(This message refers to using C-Kermit 6.0.192 under VMS V7.1)

I have some questions about how C-Kermit handles its internal buffers,
because I'm seeing some behavior that suggests it is more complex than I
thought.  In particular, I see in the C-Kermit help that there are separate
CLEAR command for DEVICE and INPUT, and I guess what I could use is some
information on how these two buffers are related -- that is, how does
information get moved from one to the other, and which one does LOG SESSION
read out of?

Basically, I'm trying to use C-Kermit to transmit some HTML form data to a
web-server, and extract a portion of the resulting returned web page.  I'm
sending the form data via the TRANSMIT command.  The web page that comes
back looks like this:

HTTP/1.1 200 OK
blah blah blah<p>
blah blah blah<p>
<font size="7"...<p>
THE DATA I WANT TO CAPTURE<p>
blah blah blah<p>
blah blah blah<p>
</html>

What I want is to capture the "THE DATA I WANT TO CAPTURE<p>" piece in an
external file for further processing outside of Kermit.  I thought this
would be pretty easy: Just connect to the server, transmit the form data to
it, do an INPUT for the |<font size="7"| string, another INPUT to advance
to the following |<p>| string, turn on session logging, do another INPUT
for the |<p>| string, turn off logging, hangup and exit.

Well, needless to say it hasn't been this straightforward, and the results
don't even seem to be consistent. It is as if the LOG SESSION command is
working out of a buffer that is somehow different (or not synchronized)
with the one the INPUT command is using.  My actual script is down below,
but before we get to that, I've stumbled over some other strangeness as
I've tried to debug this issue, so let's go over those first.

Consider the following script:
---------------------------------------------------------------------------
set host www.qrz.com:80
set input echo off
set transmit echo off
set transmit prompt 0
clear
transmit form.txt
log session form.out
pause 10
log session
hangup
exit
---------------------------------------------------------------------------

This seems about as simple as it gets: transmit the form data, start
logging the session, wait 10 seconds for everything to come through, exit.
This should record everything that comes back.  But it doesn't.  In fact,
it doesn't record anything.  BUT, if I replace that "PAUSE 10" command with
an "INPUT 10 XXXXXXX" string, THEN it works.

Why is this?  Why the necessity for the INPUT line? I assume it is to force
C-Kermit to somehow "consume" or "process" the information, but is there
any other, cleaner way to do that?  (And is this the process of moving
information between the aformentioned DEVICE and INPUT buffers?)

OK, on to my real problem.  Consider the script below, the one to solve the
problem I describe at the start of my message:

---------------------------------------------------------------------------
set host www.qrz.com:80
set input echo off
set transmit echo off
set transmit prompt 0
clear
transmit form.txt
input 10 <font size="7"
input 10 <p>
log session form.out
input 10 <p>
log session
input 10 </html>
hangup
exit
---------------------------------------------------------------------------

This doesn't work reliably, though it does work sometimes.  If I run this
script six times, sometimes I get the data I want in the FORM.OUT
file, and the other two times I get nothing.  I've tried putting debugging
statements in (to see, for example, whether the "INPUT 10 <p>" command is
failing, and sometimes it is), but they haven't helped me figure out what
is going on here.  I'm fairly confident that the remote side is returning
the same information each time, so I'm thinking the problem has to do with
the timing of exactly WHEN information from the remote side arrives.
Should those subsequent INPUT command actually be REINPUT commands?  What
happens if I have the following commands in a script:

  INPUT 10 AAA
  PAUSE 1
  INPUT 10 BBB

and the string "BBB" comes in during the first 10 seconds.  Will
that second INPUT command succeed or fail?

Final question: notice how I put that "INPUT 10 </html>" command at the end
of the script to flush out whatever other data comes back from the remote
server following the part that I am intersted in?  If I don't do this, when
I hit the EXIT at the end of the script (even though it follows the HANGUP
command), I get the "There may still be a connection open, OK to exit?"
message when the script finishes.  I assume this is because there is
"unconsumed" data in the buffer.  How do I tell Kermit to drop the
connection regardless of whether there is anything left in the buffer or
not, even if data is still streaming in?

BTW, the FORM.TXT file that I'm transmitting to the remote server looks
like this:
---------------------------------------------------------------------------
POST /wrad/directory.cgi HTTP/1.0
Referer: http://www.qrz.com/
Connection: Keep-Alive
User-Agent: Mozilla/3.0 (Macintosh; I; PPC)
Host: www.qrz.com
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, *.*
Content-type: application/x-www-form-urlencoded
Content-length: 14

callsign=k1nor
---------------------------------------------------------------------------

I know I'm asking kind of complicated questions, and I'm having some
trouble making them as straightforward as possible, but though I've been
scripting Kermit for years and years, the buffer operations and
interactions between the INPUT, REINPUT, CLEAR DEVICE, and CLEAR INPUT
commands continues to mess me up.  If someone could give me a quick rundown
on how these things interact with each other and with the LOG SESSION
command, I would be most grateful.  That alone might be enough to let me
solve my own problems with the above scripts.

- John W. Manly  <[e-mail]>